{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "46f32ff1",
   "metadata": {},
   "source": [
    "# Lesson 2: Tool Calling"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fb345ad0",
   "metadata": {},
   "source": [
    "## Setup"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "c5bbf530-3f05-434c-a70f-ac2cc4b8f7aa",
   "metadata": {
    "height": 46,
    "tags": []
   },
   "outputs": [],
   "source": [
    "from helper import get_openai_api_key\n",
    "OPENAI_API_KEY = get_openai_api_key()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "d4c06c95-e8b2-4574-b14d-685876aa1c47",
   "metadata": {
    "height": 46,
    "tags": []
   },
   "outputs": [],
   "source": [
    "import nest_asyncio\n",
    "nest_asyncio.apply()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2e53c064",
   "metadata": {},
   "source": [
    "## 1. Define a Simple Tool"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "071b717a-93cc-4332-b357-59a693359563",
   "metadata": {
    "height": 233,
    "tags": []
   },
   "outputs": [],
   "source": [
    "from llama_index.core.tools import FunctionTool\n",
    "\n",
    "def add(x: int, y: int) -> int:\n",
    "    \"\"\"Adds two integers together.\"\"\"\n",
    "    return x + y\n",
    "\n",
    "def mystery(x: int, y: int) -> int: \n",
    "    \"\"\"Mystery function that operates on top of two numbers.\"\"\"\n",
    "    return (x + y) * (x + y)\n",
    "\n",
    "\n",
    "add_tool = FunctionTool.from_defaults(fn=add)\n",
    "mystery_tool = FunctionTool.from_defaults(fn=mystery)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "d4e62118-992b-4629-9022-be8c628209c1",
   "metadata": {
    "height": 165,
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "=== Calling Function ===\n",
      "Calling function: mystery with args: {\"x\": 2, \"y\": 9}\n",
      "=== Function Output ===\n",
      "121\n",
      "121\n"
     ]
    }
   ],
   "source": [
    "from llama_index.llms.openai import OpenAI\n",
    "\n",
    "llm = OpenAI(model=\"gpt-3.5-turbo\")\n",
    "response = llm.predict_and_call(\n",
    "    [add_tool, mystery_tool], \n",
    "    \"Tell me the output of the mystery function on 2 and 9\", \n",
    "    verbose=True\n",
    ")\n",
    "print(str(response))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8cb8a835",
   "metadata": {},
   "source": [
    "## 2. Define an Auto-Retrieval Tool"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6589123f",
   "metadata": {},
   "source": [
    "### Load Data"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bcdea238",
   "metadata": {
    "tags": []
   },
   "source": [
    "To download this paper, below is the needed code:\n",
    "\n",
    "#!wget \"https://openreview.net/pdf?id=VtmBAGCN7o\" -O metagpt.pdf\n",
    "\n",
    "**Note**: The pdf file is included with this lesson. To access it, go to the `File` menu and select`Open...`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "fbe9326c-d7b3-452b-ae52-12f000157be4",
   "metadata": {
    "height": 63,
    "tags": []
   },
   "outputs": [],
   "source": [
    "from llama_index.core import SimpleDirectoryReader\n",
    "# load documents\n",
    "documents = SimpleDirectoryReader(input_files=[\"metagpt.pdf\"]).load_data()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "5451f0a3-d0a6-4b5c-a337-8e1a343ff5f0",
   "metadata": {
    "height": 63,
    "tags": []
   },
   "outputs": [],
   "source": [
    "from llama_index.core.node_parser import SentenceSplitter\n",
    "splitter = SentenceSplitter(chunk_size=1024)\n",
    "nodes = splitter.get_nodes_from_documents(documents)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "e0fe0a9c-1f87-4ae7-a79e-7c3cf9c395ed",
   "metadata": {
    "height": 29,
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "page_label: 1\n",
      "file_name: metagpt.pdf\n",
      "file_path: metagpt.pdf\n",
      "file_type: application/pdf\n",
      "file_size: 16911937\n",
      "creation_date: 2024-11-25\n",
      "last_modified_date: 2024-06-24\n",
      "\n",
      "Preprint\n",
      "METAGPT: M ETA PROGRAMMING FOR A\n",
      "MULTI -AGENT COLLABORATIVE FRAMEWORK\n",
      "Sirui Hong1∗, Mingchen Zhuge2∗, Jonathan Chen1, Xiawu Zheng3, Yuheng Cheng4,\n",
      "Ceyao Zhang4,Jinlin Wang1,Zili Wang ,Steven Ka Shing Yau5,Zijuan Lin4,\n",
      "Liyang Zhou6,Chenyu Ran1,Lingfeng Xiao1,7,Chenglin Wu1†,J¨urgen Schmidhuber2,8\n",
      "1DeepWisdom,2AI Initiative, King Abdullah University of Science and Technology,\n",
      "3Xiamen University,4The Chinese University of Hong Kong, Shenzhen,\n",
      "5Nanjing University,6University of Pennsylvania,\n",
      "7University of California, Berkeley,8The Swiss AI Lab IDSIA/USI/SUPSI\n",
      "ABSTRACT\n",
      "Remarkable progress has been made on automated problem solving through so-\n",
      "cieties of agents based on large language models (LLMs). Existing LLM-based\n",
      "multi-agent systems can already solve simple dialogue tasks. Solutions to more\n",
      "complex tasks, however, are complicated through logic inconsistencies due to\n",
      "cascading hallucinations caused by naively chaining LLMs. Here we introduce\n",
      "MetaGPT, an innovative meta-programming framework incorporating efficient\n",
      "human workflows into LLM-based multi-agent collaborations. MetaGPT en-\n",
      "codes Standardized Operating Procedures (SOPs) into prompt sequences for more\n",
      "streamlined workflows, thus allowing agents with human-like domain expertise\n",
      "to verify intermediate results and reduce errors. MetaGPT utilizes an assembly\n",
      "line paradigm to assign diverse roles to various agents, efficiently breaking down\n",
      "complex tasks into subtasks involving many agents working together. On col-\n",
      "laborative software engineering benchmarks, MetaGPT generates more coherent\n",
      "solutions than previous chat-based multi-agent systems. Our project can be found\n",
      "at https://github.com/geekan/MetaGPT.\n",
      "1 I NTRODUCTION\n",
      "Autonomous agents utilizing Large Language Models (LLMs) offer promising opportunities to en-\n",
      "hance and replicate human workflows. In real-world applications, however, existing systems (Park\n",
      "et al., 2023; Zhuge et al., 2023; Cai et al., 2023; Wang et al., 2023c; Li et al., 2023; Du et al., 2023;\n",
      "Liang et al., 2023; Hao et al., 2023) tend to oversimplify the complexities. They struggle to achieve\n",
      "effective, coherent, and accurate problem-solving processes, particularly when there is a need for\n",
      "meaningful collaborative interaction (Chen et al., 2024; Zhang et al., 2023; Dong et al., 2023; Zhou\n",
      "et al., 2023; Qian et al., 2023).\n",
      "Through extensive collaborative practice, humans have developed widely accepted Standardized\n",
      "Operating Procedures (SOPs) across various domains (Belbin, 2012; Manifesto, 2001; DeMarco &\n",
      "Lister, 2013). These SOPs play a critical role in supporting task decomposition and effective coor-\n",
      "dination. Furthermore, SOPs outline the responsibilities of each team member, while establishing\n",
      "standards for intermediate outputs. Well-defined SOPs improve the consistent and accurate exe-\n",
      "cution of tasks that align with defined roles and quality standards (Belbin, 2012; Manifesto, 2001;\n",
      "DeMarco & Lister, 2013; Wooldridge & Jennings, 1998). For instance, in a software company,\n",
      "Product Managers analyze competition and user needs to create Product Requirements Documents\n",
      "(PRDs) using a standardized structure, to guide the developmental process.\n",
      "Inspired by such ideas, we design a promising GPT -based Meta -Programming framework called\n",
      "MetaGPT that significantly benefits from SOPs. Unlike other works (Li et al., 2023; Qian et al.,\n",
      "2023), MetaGPT requires agents to generate structured outputs, such as high-quality requirements\n",
      "∗These authors contributed equally to this work.\n",
      "†Chenglin Wu (alexanderwu@fuzhi.ai) is the corresponding author, affiliated with DeepWisdom.\n",
      "1\n"
     ]
    }
   ],
   "source": [
    "print(nodes[0].get_content(metadata_mode=\"all\"))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "d7965cba-67b8-4cca-8e5f-2b0dbc96f6b0",
   "metadata": {
    "height": 80,
    "tags": []
   },
   "outputs": [],
   "source": [
    "from llama_index.core import VectorStoreIndex\n",
    "\n",
    "vector_index = VectorStoreIndex(nodes)\n",
    "query_engine = vector_index.as_query_engine(similarity_top_k=2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "560f319c-8479-40c5-9b55-480fef98deb7",
   "metadata": {
    "height": 250,
    "tags": []
   },
   "outputs": [],
   "source": [
    "from llama_index.core.vector_stores import MetadataFilters\n",
    "\n",
    "query_engine = vector_index.as_query_engine(\n",
    "    similarity_top_k=2,\n",
    "    filters=MetadataFilters.from_dicts(\n",
    "        [\n",
    "            {\"key\": \"page_label\", \"value\": \"2\"}\n",
    "        ]\n",
    "    )\n",
    ")\n",
    "\n",
    "response = query_engine.query(\n",
    "    \"What are some high-level results of MetaGPT?\", \n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "85a3091c",
   "metadata": {
    "height": 29
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Response(response='Some high-level results of MetaGPT include achieving a new state-of-the-art in code generation benchmarks with high Pass@1 scores, outperforming other popular frameworks in creating complex software projects, handling higher levels of software complexity effectively, offering extensive functionality, and demonstrating robustness and efficiency with a 100% task completion rate in experimental evaluations.', source_nodes=[NodeWithScore(node=TextNode(id_='59a827f7-008e-4224-ac30-127466da6818', embedding=None, metadata={'page_label': '2', 'file_name': 'metagpt.pdf', 'file_path': 'metagpt.pdf', 'file_type': 'application/pdf', 'file_size': 16911937, 'creation_date': '2024-11-25', 'last_modified_date': '2024-06-24'}, excluded_embed_metadata_keys=['file_name', 'file_type', 'file_size', 'creation_date', 'last_modified_date', 'last_accessed_date'], excluded_llm_metadata_keys=['file_name', 'file_type', 'file_size', 'creation_date', 'last_modified_date', 'last_accessed_date'], relationships={<NodeRelationship.SOURCE: '1'>: RelatedNodeInfo(node_id='108abb5d-2de5-4876-bb4a-09f1b55de543', node_type=<ObjectType.DOCUMENT: '4'>, metadata={'page_label': '2', 'file_name': 'metagpt.pdf', 'file_path': 'metagpt.pdf', 'file_type': 'application/pdf', 'file_size': 16911937, 'creation_date': '2024-11-25', 'last_modified_date': '2024-06-24'}, hash='9fa2093ee11de8c10ce82694113f39f3e5bdf6e05ac8195b92dffb6902d926fa'), <NodeRelationship.PREVIOUS: '2'>: RelatedNodeInfo(node_id='0d843c5e-39fb-40f2-990f-6bd6f1aea65e', node_type=<ObjectType.TEXT: '1'>, metadata={'page_label': '1', 'file_name': 'metagpt.pdf', 'file_path': 'metagpt.pdf', 'file_type': 'application/pdf', 'file_size': 16911937, 'creation_date': '2024-11-25', 'last_modified_date': '2024-06-24'}, hash='fbf95cf5a23181f54ac4dcdea1a2fc92e4f5b40c5c81190b0bbc69ac59a28b55'), <NodeRelationship.NEXT: '3'>: RelatedNodeInfo(node_id='c369d4b9-d3ca-499a-be81-d23f193a28a7', node_type=<ObjectType.TEXT: '1'>, metadata={}, hash='7622cc60ba07cb07012cdc90d6a4eabd364da5fb76b2c5fd02e1931382e9258b')}, text='Preprint\\nFigure 1: The software development SOPs between MetaGPT and real-world human teams.\\nIn software engineering, SOPs promote collaboration among various roles. MetaGPT showcases\\nits ability to decompose complex tasks into specific actionable procedures assigned to various roles\\n(e.g., Product Manager, Architect, Engineer, etc.).\\ndocuments, design artifacts, flowcharts, and interface specifications. The use of intermediate struc-\\ntured outputs significantly increases the success rate of target code generation. Because it helps\\nmaintain consistency in communication, minimizing ambiguities and errors during collaboration.\\nMore graphically, in a company simulated by MetaGPT, all employees follow a strict and stream-\\nlined workflow, and all their handovers must comply with certain established standards. This reduces\\nthe risk of hallucinations caused by idle chatter between LLMs, particularly in role-playing frame-\\nworks, like: “ Hi, hello and how are you?” – Alice (Product Manager); “ Great! Have you had\\nlunch?” – Bob (Architect).\\nBenefiting from SOPs, MetaGPT offers a promising approach to meta-programming. In this context,\\nwe adopt meta-programming1as ”programming to program”, in contrast to the broader fields of meta\\nlearning and ”learning to learn” (Schmidhuber, 1987; 1993a; Hochreiter et al., 2001; Schmidhuber,\\n2006; Finn et al., 2017).\\nThis notion of meta-programming also encompasses earlier efforts like CodeBERT (Feng et al.,\\n2020) and recent projects such as CodeLlama (Rozi `ere et al., 2023) and WizardCoder (Luo\\net al., 2023). However, MetaGPT stands out as a unique solution that allows for efficient meta-\\nprogramming through a well-organized group of specialized agents. Each agent has a specific role\\nand expertise, following some established standards. This allows for automatic requirement analysis,\\nsystem design, code generation, modification, execution, and debugging during runtime, highlight-\\ning how agent-based techniques can enhance meta-programming.\\nTo validate the design of MetaGPT, we use publicly available HumanEval (Chen et al., 2021a) and\\nMBPP (Austin et al., 2021) for evaluations. Notably, in code generation benchmarks, MetaGPT\\nachieves a new state-of-the-art (SoTA) with 85.9% and 87.7% in Pass@1. When compared to other\\npopular frameworks for creating complex software projects, such as AutoGPT (Torantulino et al.,\\n2023), LangChain (Chase, 2022), AgentVerse (Chen et al., 2023), and ChatDev (Qian et al., 2023).\\nMetaGPT also stands out in handling higher levels of software complexity and offering extensive\\nfunctionality. Remarkably, in our experimental evaluations, MetaGPT achieves a 100% task com-\\npletion rate, demonstrating the robustness and efficiency (time and token costs) of our design.\\nWe summarize our contributions as follows:\\n1https://en.wikipedia.org/w/index.php?title=Metaprogramming\\n2', start_char_idx=0, end_char_idx=2866, text_template='{metadata_str}\\n\\n{content}', metadata_template='{key}: {value}', metadata_seperator='\\n'), score=0.8375392682876993)], metadata={'59a827f7-008e-4224-ac30-127466da6818': {'page_label': '2', 'file_name': 'metagpt.pdf', 'file_path': 'metagpt.pdf', 'file_type': 'application/pdf', 'file_size': 16911937, 'creation_date': '2024-11-25', 'last_modified_date': '2024-06-24'}})"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "response"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "2da4042f-8fdb-4959-8760-86685c903cfd",
   "metadata": {
    "height": 29,
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Some high-level results of MetaGPT include achieving a new state-of-the-art in code generation benchmarks with high Pass@1 scores, outperforming other popular frameworks in creating complex software projects, handling higher levels of software complexity effectively, offering extensive functionality, and demonstrating robustness and efficiency with a 100% task completion rate in experimental evaluations.\n"
     ]
    }
   ],
   "source": [
    "print(str(response))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "ecdb53aa",
   "metadata": {
    "height": 29
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(response.source_nodes)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "30bb264c-42e0-46f8-9d28-da11a8535960",
   "metadata": {
    "height": 46,
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'page_label': '2', 'file_name': 'metagpt.pdf', 'file_path': 'metagpt.pdf', 'file_type': 'application/pdf', 'file_size': 16911937, 'creation_date': '2024-11-25', 'last_modified_date': '2024-06-24'}\n"
     ]
    }
   ],
   "source": [
    "for n in response.source_nodes:\n",
    "    print(n.metadata)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6c392482",
   "metadata": {},
   "source": [
    "### Define the Auto-Retrieval Tool"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "2639e79b-f615-425b-85ea-8a279bb26dd0",
   "metadata": {
    "height": 621,
    "tags": []
   },
   "outputs": [],
   "source": [
    "from typing import List\n",
    "from llama_index.core.vector_stores import FilterCondition\n",
    "\n",
    "\n",
    "def vector_query(\n",
    "    query: str, \n",
    "    page_numbers: List[str]\n",
    ") -> str:\n",
    "    \"\"\"Perform a vector search over an index.\n",
    "    \n",
    "    query (str): the string query to be embedded.\n",
    "    page_numbers (List[str]): Filter by set of pages. Leave BLANK if we want to perform a vector search\n",
    "        over all pages. Otherwise, filter by the set of specified pages.\n",
    "    \n",
    "    \"\"\"\n",
    "\n",
    "    metadata_dicts = [\n",
    "        {\"key\": \"page_label\", \"value\": p} for p in page_numbers\n",
    "    ]\n",
    "    \n",
    "    query_engine = vector_index.as_query_engine(\n",
    "        similarity_top_k=2,\n",
    "        filters=MetadataFilters.from_dicts(\n",
    "            metadata_dicts,\n",
    "            condition=FilterCondition.OR\n",
    "        )\n",
    "    )\n",
    "    response = query_engine.query(query)\n",
    "    return response\n",
    "    \n",
    "\n",
    "vector_query_tool = FunctionTool.from_defaults(\n",
    "    name=\"vector_tool\",\n",
    "    fn=vector_query\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "2a408ace-cf25-425b-8248-7028ceabcd42",
   "metadata": {
    "height": 114,
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "=== Calling Function ===\n",
      "Calling function: vector_tool with args: {\"query\": \"high-level results of MetaGPT\", \"page_numbers\": [\"2\"]}\n",
      "=== Function Output ===\n",
      "MetaGPT achieves a new state-of-the-art (SoTA) in code generation benchmarks with 85.9% and 87.7% in Pass@1. It stands out in handling higher levels of software complexity and offering extensive functionality. In experimental evaluations, MetaGPT achieves a 100% task completion rate, demonstrating robustness and efficiency in terms of time and token costs.\n"
     ]
    }
   ],
   "source": [
    "llm = OpenAI(model=\"gpt-3.5-turbo\", temperature=0)\n",
    "response = llm.predict_and_call(\n",
    "    [vector_query_tool], \n",
    "    \"What are the high-level results of MetaGPT as described on page 2?\", \n",
    "    verbose=True\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "6ec05565-6adf-4294-ba5c-b384220876ac",
   "metadata": {
    "height": 46,
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'page_label': '2', 'file_name': 'metagpt.pdf', 'file_path': 'metagpt.pdf', 'file_type': 'application/pdf', 'file_size': 16911937, 'creation_date': '2024-11-25', 'last_modified_date': '2024-06-24'}\n"
     ]
    }
   ],
   "source": [
    "for n in response.source_nodes:\n",
    "    print(n.metadata)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fef4dec0",
   "metadata": {},
   "source": [
    "## Let's add some other tools!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "55dd32e5-e29f-42ed-839a-ca937fe4743e",
   "metadata": {
    "height": 267,
    "tags": []
   },
   "outputs": [],
   "source": [
    "from llama_index.core import SummaryIndex\n",
    "from llama_index.core.tools import QueryEngineTool\n",
    "\n",
    "summary_index = SummaryIndex(nodes)\n",
    "summary_query_engine = summary_index.as_query_engine(\n",
    "    response_mode=\"tree_summarize\",\n",
    "    use_async=True,\n",
    ")\n",
    "summary_tool = QueryEngineTool.from_defaults(\n",
    "    name=\"summary_tool\",\n",
    "    query_engine=summary_query_engine,\n",
    "    description=(\n",
    "        \"Useful if you want to get a summary of MetaGPT\"\n",
    "    ),\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "4228ca7c-42a0-494b-987b-5a1c5c584536",
   "metadata": {
    "height": 97,
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "=== Calling Function ===\n",
      "Calling function: vector_tool with args: {\"query\": \"MetaGPT comparisons with ChatDev\", \"page_numbers\": [\"8\"]}\n",
      "=== Function Output ===\n",
      "MetaGPT outperforms ChatDev on the SoftwareDev dataset in various metrics. For example, MetaGPT achieves a higher score in executability, takes less time for execution, uses more tokens but requires fewer tokens to generate one line of code compared to ChatDev. Additionally, MetaGPT surpasses ChatDev in code statistics and the cost of human revision. These results emphasize the advantages of utilizing SOPs in collaborative efforts between multiple agents.\n"
     ]
    }
   ],
   "source": [
    "response = llm.predict_and_call(\n",
    "    [vector_query_tool, summary_tool], \n",
    "    \"What are the MetaGPT comparisons with ChatDev described on page 8?\", \n",
    "    verbose=True\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "id": "4aa3e8c1-a8c3-4c92-a0e4-5c081f91d966",
   "metadata": {
    "height": 46,
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'page_label': '8', 'file_name': 'metagpt.pdf', 'file_path': 'metagpt.pdf', 'file_type': 'application/pdf', 'file_size': 16911937, 'creation_date': '2024-11-25', 'last_modified_date': '2024-06-24'}\n"
     ]
    }
   ],
   "source": [
    "for n in response.source_nodes:\n",
    "    print(n.metadata)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "21906d47-7266-4479-bbb4-9f392d5c399b",
   "metadata": {
    "height": 97,
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "=== Calling Function ===\n",
      "Calling function: summary_tool with args: {\"input\": \"This paper discusses the impact of climate change on biodiversity and ecosystems.\"}\n"
     ]
    }
   ],
   "source": [
    "response = llm.predict_and_call(\n",
    "    [vector_query_tool, summary_tool], \n",
    "    \"What is a summary of the paper?\", \n",
    "    verbose=True\n",
    ")"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
